UnexpectedRowsExpectation unexpected_rows limit

GX Core Support
1

Hi! I’m using this UnexpectedRowsExpectation expectation and I need to retrieve ALL the rows that has failed the condition but, unexpected_rows only has a sample of 200 rows. Is any configuration to allow this expectation to retrieve all rows that has failed?
Doesn’t make any sense that this is limited to 200.
To get ALL unexpected rows I have to query again the table I’m validating?

Config:

{
 "result_format": "COMPLETE",
 "unexpected_index_column_names": [col.strip() for col in row["table_key"].split(",")],
 "partial_unexpected_count": 0,
 "exclude_unexpected_values": False,
 "include_unexpected_rows": True,
 "return_unexpected_index_query": False,
}

Thanks in advance :slight_smile:

2

I saw your have a constant (MAX_RESULT_RECORDS) that is hardcoded to 200:
great_expectations/great_expectations/expectations/metrics/util.py

and is limiting the query output here:

There is any way that I can change this value? Would be useful to enable or disable this, using -1 or N (N could be 200 by default), via an environment variable, for example, as this one: os.environ["GX_ANALYTICS_ENABLED"] = "false"
or just adding an argument to the Expectation.

3

Hi there,

As you’ve observed, the maximum number of records is currently set to 200. This limitation is intentional since failure scenarios can generate hundreds or even thousands of rows, which might be overwhelming to handle.

If customizing or increasing this limit is crucial for your use case, I recommend submitting a feature request to see if this functionality is something we could consider.

Thank you