Hi,
I am using several type of expectations ( null count , a value has to be within a range etc.). Now , I have a requirement to compare current result of each expectation with previous run and pass/fail the comparison in case difference between current and previous run is beyond a defined percentage variance. Example - Null count of column for latest run is 10 and Null count of column for previous run is 6. So, percentage difference between these two executions is ( (10 - 6)/10 ) * 100 = 4% . Now, we have defined threshold percentage (example : 5%) . So, if percentage difference between these two executions (4%) is more than defined threshold percentage (example : 5%) then “Null count” expectation status for the current run will be considered as FAIL.
Q) Is there anyway , we can do this comparison of results between two executions ?
Or, Is there anyway to pass previous result to current expectation execution and perform percentage variance calculation as I shared in above example?
Great Expectations does not provide built-in functionality to compare current and previous validation results directly within an expectation, you can achieve this behavior with some type of custom validation flow where you pull results out and compute