Was requested to post this here.
Hey, I made an issue ticket for something I encountered: DataContext, BaseDataContext, DataContextConfig, and get_checkpoint() · Issue #1795 · great-expectations/great_expectations · GitHub.
It has to do with DataContext and DataContextConfig . Long story short, is there any way to dynamically generate a DataContext via the API (as shown in the docs How to instantiate a Data Context on an EMR Spark cluster — great_expectations documentation) in order to run a Checkpoint? It seems that right now only yaml configs are accepted.
1 Like
A BaseDataContext that you instantiate programmatically without a configuration file (as described in the how-to guide linked in the question) indeed does not have the capability to run Checkpoints.
Since Checkpoint is a thin wrapper around ValidationOperators, you can run a ValidationOperator by invoking the context’s run_validation_operator method.
This folder in the repo has Jupyter notebooks that show step by step how to prepare the arguments to run a ValidationOperator: https://github.com/great-expectations/great_expectations/tree/develop/great_expectations/init_notebooks (there are versions for Pandas, Spark and SQLAlchemy)
1 Like