After trying to replicate the tutorials on creating a custom Query expectation, copying the already implemented ones, I find myself unable to import any Query Expectation into my GX suite. Even just adding the already shipped ones in my GX suite (like “expect_queried_column_value_frequency_to_meet_threshold”) just yields the Error
‘ValueError: Unable to determine “domain_type” of this “ExpectationConfiguration” object from “kwargs” and heuristics.’
I am using GX 18.3,Python 3.11 with Pyspark on a small data frame. My Code otherwise works if I don’t use the any Query Metrics.
Hi @mant to get custom expectations to get imported, you need to follow few steps.
Create your context using this code:
import great_expectations as gx
from great_expectations.core.expectation_configuration import ExpectationConfiguration
import pandas as pd
import os
from great_expectations.data_context import FileDataContext
base = os.getcwd()
path_to_empty_folder = base + "/my_gx_project/"
context = FileDataContext.create(project_root_dir=path_to_empty_folder)
Put your custom expectations code inside great_expectations → plugins → expectations
Then run from my_gx_project.great_expectations.plugins.expectations import your_custom_expectation. This is if your GX notebook is in the same root as the project folder (I called it “my_gx_project”).
Remember that context needs to be called before importing custom expectations.
Thanks for your answer, but this is not my problem.
I cannot add QUERY expectations to my suite. I followed the tutorial on creating CUSTOM QUERY EXPECTATIONS. I add them to my plugins folder and then import them. Then I get the error
Traceback (most recent call last):
File "/Users/christoph/PycharmProjects/great_expectations_test/gx_cli_test_pyspark.py", line 33, in <module>
my_suite.show_expectations_by_expectation_type()
File "/Users/christoph/PycharmProjects/great_expectations_test/.venv/lib/python3.11/site-packages/great_expectations/core/expectation_suite.py", line 798, in show_expectations_by_expectation_type
self.get_grouped_and_ordered_expectations_by_expectation_type()
File "/Users/christoph/PycharmProjects/great_expectations_test/.venv/lib/python3.11/site-packages/great_expectations/core/expectation_suite.py", line 861, in get_grouped_and_ordered_expectations_by_expectation_type
self.get_table_expectations(),
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/christoph/PycharmProjects/great_expectations_test/.venv/lib/python3.11/site-packages/great_expectations/core/expectation_suite.py", line 885, in get_table_expectations
expectation_configurations: List[ExpectationConfiguration] = list(
^^^^^
File "/Users/christoph/PycharmProjects/great_expectations_test/.venv/lib/python3.11/site-packages/great_expectations/core/expectation_suite.py", line 887, in <lambda>
lambda element: element.get_domain_type() == MetricDomainTypes.TABLE,
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/christoph/PycharmProjects/great_expectations_test/.venv/lib/python3.11/site-packages/great_expectations/core/expectation_configuration.py", line 1514, in get_domain_type
raise ValueError(
ValueError: Unable to determine "domain_type" of this "ExpectationConfiguration" object from "kwargs" and heuristics.```
This happens with ALL QUERY expectations in Great Expectations.
This is what I add to my Suite.json file