A user asked this question on Slack - reposting it here, since others might run into this issue as well.
The user wanted to provide the database username and password to GE using env variables (following this doc: https://docs.greatexpectations.io/en/latest/reference/data_context_reference.html#managing-environment-and-secrets)
The user used the ${…} syntax in the uncommitted/config_variables.yml, but GE would not connect to the database and it was clear that the env vars are not being substituted:
Here is why this did not work:
Data Context configuration supports the substitution syntax only in the great_expectations.yml config file, so any ${…} in config_variables.yml is not going to be substituted.
If you want to pass the database password to a datasource via an env var, modify this datasource’s config block in great_expectations.yml:
Before:
datasources:
datawarehouse:
class_name: SqlAlchemyDatasource
data_asset_type:
class_name: SqlAlchemyDataset
module_name:
credentials: ${datawarehouse}
After:
datasources:
datawarehouse:
class_name: SqlAlchemyDatasource
data_asset_type:
class_name: SqlAlchemyDataset
module_name:
credentials:
drivername: postgresql+psycopg2
host: myhost.foo
port: '5439'
database: mydatabase
username: ${DATAWAREHOUSE_USERNAME}
password: ${DATAWAREHOUSE_PASSWORD}
(DATAWAREHOUSE_USERNAME and DATAWAREHOUSE_PASSWORD are env variables)