Hi,
I am setting up an expectation to check Column-A’s values to be not null with a condition that column-B’s value is XYZ. Details are below.
However, I am experiencing an exception while running a row condition on the expectation that checks the data on the Redshift table’s column.
This exception only occurs when either column name or column value contains space.
Previously, this issue was raised on GitHub, but this was only resolved partially.
Could you please help how can we use or parse the expectation to work with row_condition that has space in either values or column names??
Great Expectation Version: version 0.16.15
Query Engine: SqlAlchemyExecutionEngine
{
"expectation_type": "expect_column_values_to_not_be_null",
"kwargs": {
"column": "se years business in operation",
"row_condition": "col(\"\"employment basis\"\")==\"\"Self Employed\"\"",
"condition_parser": "great_expectations__experimental__"
},
"meta": {}
},
Exception details
"exception_message": "unable to parse condition: col(\"\"borrower employment\"\")==\"\"Self Employed\"\"",
"exception_traceback": "Traceback (most recent call last):
File \"/useraccount/.pyenv/versions/3.11.0/lib/python3.11/site-packages/great_expectations/expectations/row_conditions.py\", line 121, in _parse_great_expectations_condition
return condition.parseString(row_condition)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File \"/useraccount/.pyenv/versions/3.11.0/lib/python3.11/site-packages/pyparsing/core.py\", line 1141, in parse_string
raise exc.with_traceback(None)
pyparsing.exceptions.ParseException: Expected {{Combine:({Suppress:('col(\"') W:(A-Za-z, -.0-9A-Z_a-z) Suppress:('\")')}) '.NOTNULL()'} ^ {Combine:({Suppress:('col(\"') W:(A-Za-z, -.0-9A-Z_a-z) Suppress:('\")')}) {'>' ^ '<' ^ '>=' ^ '<=' ^ '==' ^ '!='} {Re:('[+-]?\\d+(?:\\.\\d*)?(?:[eE][+-]?\\d+)?') ^ {Suppress:('\"') W:(\t !#-&(-~) Suppress:('\"')} ^ {Suppress:(\"'\") W:(\t !#-&(-~) Suppress:(\"'\")} ^ {'date' Suppress:('(') {{Suppress:('\"') W:(\t !#-&(-~) Suppress:('\"')} ^ {Suppress:(\"'\") W:(\t !#-&(-~) Suppress:(\"'\")}} Suppress:(')')}}}}, found '\"' (at char 5), (line:1, col:6)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File \"/useraccount/.pyenv/versions/3.11.0/lib/python3.11/site-packages/great_expectations/execution_engine/execution_engine.py\", line 560, in _process_direct_and_bundled_metric_computation_configurations
] = self.resolve_metric_bundle(
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File \"/useraccount/.pyenv/versions/3.11.0/lib/python3.11/site-packages/great_expectations/execution_engine/sqlalchemy_execution_engine.py\", line 1083, in resolve_metric_bundle
selectable: sqlalchemy.Selectable = self.get_domain_records(
^^^^^^^^^^^^^^^^^^^^^^^^
File \"/useraccount/.pyenv/versions/3.11.0/lib/python3.11/site-packages/great_expectations/execution_engine/sqlalchemy_execution_engine.py\", line 708, in get_domain_records
parsed_condition = parse_condition_to_sqlalchemy(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File \"/useraccount/.pyenv/versions/3.11.0/lib/python3.11/site-packages/great_expectations/expectations/row_conditions.py\", line 176, in parse_condition_to_sqlalchemy
parsed = _parse_great_expectations_condition(row_condition)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File \"/useraccount/.pyenv/versions/3.11.0/lib/python3.11/site-packages/great_expectations/expectations/row_conditions.py\", line 123, in _parse_great_expectations_condition
raise ConditionParserError(f\"unable to parse condition: {row_condition}\")
great_expectations.expectations.row_conditions.ConditionParserError: unable to parse condition: col(\"\"borrower employment\"\")==\"\"Self Employed\"\"
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File \"/useraccount/.pyenv/versions/3.11.0/lib/python3.11/site-packages/great_expectations/validator/validation_graph.py\", line 276, in _resolve
self._execution_engine.resolve_metrics(
File \"/useraccount/.pyenv/versions/3.11.0/lib/python3.11/site-packages/great_expectations/execution_engine/execution_engine.py\", line 280, in resolve_metrics
return self._process_direct_and_bundled_metric_computation_configurations(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File \"/useraccount/.pyenv/versions/3.11.0/lib/python3.11/site-packages/great_expectations/execution_engine/execution_engine.py\", line 565, in _process_direct_and_bundled_metric_computation_configurations
raise gx_exceptions.MetricResolutionError(
great_expectations.exceptions.exceptions.MetricResolutionError: unable to parse condition: col(\"\"employment basis\"\")==\"\"Self Employed\"\"",