Hi,
I’m using acryl-datahub-gx-plugin==1.3.1 for Great Expectations validation and pushing the results/metadata to DataHub.
As per the documentation here, I’m defining the row_condition like this (using SQLAlchemy with Snowflake):
row_condition = col(“<col_name>”) == “Value”
condition_parser = “great_expectations__experimental__”
This works as expected for a single condition.
However, I’m unable to apply multiple conditions in the row_condition parameter. I tried the following approaches:
row_condition = col(“col1”) == “Value” & col(“col2”) == “Value”
row_condition = (col(“col1”) == “Value”) & (col(“col2”) == “Value”)
row_condition = (col(“col1”) == “Value” & col(“col2”) == “Value”)
Behavior observed:
-
In the first case, only the first condition is applied (the second condition is ignored).
-
In the second and third cases, the validation fails.
Additionally, the plugin currently supports DataHubValidationAction only for Great Expectations versions >= 0.18.0, < 1.0.0. Because of this version constraint, I’m unable to upgrade to GX > 1.0.0.
Is there a recommended way to apply multiple conditions in row_condition when using SQLAlchemy (Snowflake) with this plugin?
Additionally, the unexpected_index_query in the validation results does not include the row_condition column filter.
Below is the runtime_configuration I am using while creating the checkpoint:
gx_result_format = {
“result_format”: “COMPLETE”,
“return_unexpected_index_query”: True
}
are these expected behavior in GX?