Wanted help in creating query Expectation to use on diffrent column name

GX OSS Support
, ,
1

Hi,

I was using the custom query expectation provided (expect_queried_column_value_frequency_to_meet_threshold.py) in the documentation, to run on different column, but when I run the checkpoint I only see the results of column2 in expectation results, where as it should have included both the columns expectation results. I’m running this on databricks with latest version of great expectation.

validator.expect_queried_column_value_frequency_to_meet_threshold(column =“column1”, value=1,threshold=1)

validator.expect_queried_column_value_frequency_to_meet_threshold(column =“column2”, value=1,threshold=1)

Is this functionality supported? if yes then how to solve it.

2

Hi @DeepakAdigourSharma this isn’t tied to custom expectation. This normally shouldn’t happen. First thing to confirm, did you run validator.save_expectation_suite() after you ran your validator.expect...?

Secondly, I would suggest using validations object rather than validators. Validators are for quick observation about your data, not meant for production usage. Can you modify your code with this template to see if it works?

import great_expectations as gx
from great_expectations.core.expectation_configuration import ExpectationConfiguration
import pandas as pd
import os 
from great_expectations.data_context import FileDataContext

base = os.getcwd()

path_to_empty_folder = base + "/my_gx_project/"

context = FileDataContext.create(project_root_dir=path_to_empty_folder)


datasource = context.sources....

asset = datasource....
batch_request = asset.build_batch_request()

ec = ExpectationConfiguration(expectation_type= 'expect_queried_column_value_frequency_to_meet_threshold', 
                              kwargs={'column': 'column1', 'value': 1, 'threshold': 1}
                             )

ec2 = ExpectationConfiguration(expectation_type= 'expect_queried_column_value_frequency_to_meet_threshold', 
                              kwargs={'column': 'column2', 'value': 1, 'threshold': 1}
                             )

suite_name = 'my_suite'

suite = context.add_expectation_suite(suite_name, expectations=[ec,ec2])

context.save_expectation_suite(expectation_suite = suite)


validations = [{"batch_request": batch_request, "expectation_suite_name": suite_name}]

checkpoint = context.add_or_update_checkpoint(
    name="sample checkpoint", validations = validations
)

run = checkpoint.run()
res = run.list_validation_result_identifiers()[0]
context.build_data_docs()
context.open_data_docs(res)
3

Thanks @HaebichanGX. It worked.

1 Like
4

Great! I’m glad the solution was successful.