I’m trying to run a checkpoint in a Python environment from a parent directory which contains a great expectations initialized suite (ie: parent_dir/great_expectations ). Even though I run os.chdir(./great_expectations) , I get an error when I am in the parent directory: AttributeError: module 'great_expectations' has no attribute 'data_context' while trying to initialize a context like so: context = ge.data_context.DataContext(parent_dir/great_expectations) . I only get this error when trying to run the checkpoint in a Python environment when I am located in parent_dir , and do not get the error when I am in parent_dir/great_expectations. Anyone experience this before? Is there any way I can run checkpoints starting from a parent directory?
My use case: I have a folder of dags which Airflow uses as the base directory. Within that folder there’s a great expectations suite. I’d like to run GE as part of a dag, which inherently uses the parent dags folder.
1 Like
@sarahperpay The current directory should not make a difference when running validation in Python, as long as the path for the directory containing the great_expectations.yml config file of the Data Context (project) is passed as an arg to ge.data_context.DataContext(directory_with_project_config_file).
From the post it looks like the call is done correctly.
The error message AttributeError: module 'great_expectations' has no attribute 'data_context' makes me think that the issue is related to how Great Expectations is installed. Is it “pip-installed” in the environment or is the repo just cloned?
If it helps, please ping us on Slack so that we can troubleshoot it together while looking at the same screen.
Quick update: I did find a solution. It looks like the great_expectations module does not import correctly from a parent directory; I have to first run os.chdir(my_dir_with_ge_yaml) and only after that run import great_expectations as ge. Then I can set the context.
EDIT: I also realized that my parent folder was named great_expectations, and the import overlapped with the actual Python module. Renaming the folder did the trick.
cc @eugene.mandel
1 Like