How to override setting in great_expectations.yml for different environment?

Archive
1

How to configure great expectations to run different settings? I have a local and production settings which has different path for their validation/expectation/doc path? In general, I have need to override some keys in the great_expectations.yml

Is config_variables.yml designed to use for these situation?

Args:
project_root_dir: path to the root directory in which to create a new great_expectations directory
runtime_environment: a dictionary of config variables that
override both those set in config_variables.yml and the environment

 Args:
            project_root_dir: path to the root directory in which to create a new great_expectations directory
            runtime_environment: a dictionary of config variables that
            override both those set in config_variables.yml and the environment

DataContext seems accept some argument, but the documents does not match the code.

I also try just passing in a dictionary according to the docstring, but it does not work as expected.

image
image1892×611 50.4 KB

2

One of the easiest way to handle this might be to use environment variables for your dev/prod environments and set them before running GE. See the docs here: https://docs.greatexpectations.io/en/latest/guides/how_to_guides/configuring_data_contexts/how_to_use_a_yaml_file_or_environment_variables_to_populate_credentials.html?highlight=environment%20variables

That way you can do something like export DB_HOST="my_prod_db_host" or export DB_HOST="my_dev_db_host" before evoking GE or the jupyter notebook.

3

thanks for getting back. I do not think environment variable is suitable for me. For me local development will be point to some offline file while for production i need to point to some db. so i will like to have a choice to use which yaml file for datacontext