What does the validation_results_limit option in Data Docs limit?

Archive
1

Does the validation_results_limit data docs config option limit the number of HTML files in the validations backend store or does it just limit the number that get included in the index? For example, if I had over 1000 validations in my validations store, would validation_results_limit: 1000 keep the number of files in data_docs/validations under 1000? and if not, is there a workaround to make sure that the website backend store doesn’t grow too large?

1 Like
2

The validation_results_limit config option limits 2 things:

  1. the number of (most recent) Validation Results the Data Docs site builder renders into an HTML file when it is called to build a Data Docs site.
  2. the number of (most recent) HTML pages of Validation Results that the index page of a Data Docs site links to

The option does not limit the total number of HTML documents in the HTML store of a Data Docs site.

Once you set the validation_results_limit config option for a Data Docs site, you can run great_expectations docs clean periodically in order to make sure that the number of of HTML documents in the site’s store does not grow infinitely.

If the option is not configured, ALL Validation Results will be rendered.

Here is the documentation article with an example of configuration: Data Docs Reference — great_expectations documentation