How to pass an environment variable to a checkpoint yml file?

Archive
1

Can you pass a date as env variable to the checkpoint yml file like this:

batches:
  - batch_kwargs:
      query: "SELECT * FROM table
              WHERE start = '$validation_date'"
      query_parameters:
        validation_date: ${VALIDATION_DATE}
      datasource: athena
      data_asset_name:
    expectation_suite_names: # one or more suites may validate against a single batch
      - athena_table

“When I run checkpoint it does not evaluate to my local env variable, although I can do echo $VALIDATION_DATE on my terminal and see my date correctly.
How do you pass query parameter to your checkpoint yml file?”

2 Likes
2

Great Expectations supports variable substitution in great_expectations.yml and config_variables.yml files, but not in the yml config files of Checkpoints.

We are working on making it possible to pass more flexible config to Checkpoints.

In the meantime, a workaround is to use the lower level API - Validation Operator. A Checkpoint is a wrapper around a Validation Operator. You should invoke a Validation Operator from Python like in this example: https://github.com/great-expectations/great_expectations/blob/develop/great_expectations/init_notebooks/sql/validation_playground.ipynb

1 Like