Data Docs rendering exception when using a query store parameter for mostly

GX Core Support
1

Hi. I’m seeing a Data Docs rendering exception when using a query store parameter for mostly.

Here is my expectation:

    {
      "expectation_type": "expect_column_values_to_not_be_null",
      "kwargs": {
        "column": "MyColumn",
        "mostly": { "$PARAMETER": "urn:great_expectations:stores:my_query_store:my_query_name" }
      }
    },

my_query_name corresponds to a query in my_query_store with "return_type": "scalar".

data_context_config = DataContextConfig(
  #...
  stores={
      "my_query_store": {
        "class_name": "SqlAlchemyQueryStore",
        "credentials": {
            #...
        },
        "queries": {
          "my_query_name": { "query": f"SELECT value FROM mytable WHERE mywhere", "return_type": "scalar" }
        }
  }
)
context = BaseDataContext(project_config=data_context_config)

The Exception raised is as follows:

An unexpected Exception occurred during data docs rendering.  Because of this error, certain parts of data docs will not be rendered properly and/or may not appear altogether.  Please use the trace, included in this message, to diagnose and repair the underlying issue.  Detailed information follows:
        TypeError: "'<' not supported between instances of 'dict' and 'float'".  Traceback: "Traceback (most recent call last):
  File "/local_disk0/.ephemeral_nfs/envs/pythonEnv-e8623e98-3a54-49b4-89e9-a77c6b5aa97b/lib/python3.9/site-packages/great_expectations/render/renderer/content_block/content_block.py", line 108, in _render_list
    result = content_block_fn(
  File "/local_disk0/.ephemeral_nfs/envs/pythonEnv-e8623e98-3a54-49b4-89e9-a77c6b5aa97b/lib/python3.9/site-packages/great_expectations/render/renderer/renderer.py", line 15, in inner_func
    return renderer_fn(*args, **kwargs)
  File "/local_disk0/.ephemeral_nfs/envs/pythonEnv-e8623e98-3a54-49b4-89e9-a77c6b5aa97b/lib/python3.9/site-packages/great_expectations/expectations/expectation.py", line 166, in inner_func
    ] = render_func(*args, **kwargs)
  File "/local_disk0/.ephemeral_nfs/envs/pythonEnv-e8623e98-3a54-49b4-89e9-a77c6b5aa97b/lib/python3.9/site-packages/great_expectations/expectations/core/expect_column_values_to_not_be_null.py", line 169, in _prescriptive_renderer
    if params["mostly"] is not None and params["mostly"] < 1.0:  # noqa: PLR2004
TypeError: '<' not supported between instances of 'dict' and 'float'
".

Reviewing the Data Docs, the Validation Results render perfectly, with the value from the query substituted correctly. It seems the rendering issue is with the Expectation Suites view, which displays as shown below:

image
image2131×84 14.2 KB

I suppose this makes sense as there is no value available to substitute for the parameter outside of a checkpoint run. However, it would be prefereable for this scenario to be handled without throwing an Exception if at all possible.

We have around 200 expectations using this pattern, so these Exceptions are creating quite a bit of noise!!

Many thanks.

2

Correct rendering in Validation Results section of Data Docs:

image
image2466×395 27.1 KB

3

Hi @mike_f50 it looks like that you’re using an older version of GX (using configs) which we no longer support. Can you please update your GX code to our latest version that uses Fluent Data Sources (FDS) and see the same problem is occurring?

To better understand FDS, you can refer to this flowchart: Miro | Online Whiteboard for Visual Collaboration

4

Thanks @HaebichanGX. I shall look into Fluent Data Sources and get back to you.