DataContext, BaseDataContext, DataContextConfig, and get_checkpoint()

Was requested to post this here.

Hey, I made an issue ticket for something I encountered: DataContext, BaseDataContext, DataContextConfig, and get_checkpoint() · Issue #1795 · great-expectations/great_expectations · GitHub.

It has to do with DataContext and DataContextConfig . Long story short, is there any way to dynamically generate a DataContext via the API (as shown in the docs How to instantiate a Data Context on an EMR Spark cluster — great_expectations documentation) in order to run a Checkpoint? It seems that right now only yaml configs are accepted.

1 Like

A BaseDataContext that you instantiate programmatically without a configuration file (as described in the how-to guide linked in the question) indeed does not have the capability to run Checkpoints.
Since Checkpoint is a thin wrapper around ValidationOperators, you can run a ValidationOperator by invoking the context’s run_validation_operator method.

This folder in the repo has Jupyter notebooks that show step by step how to prepare the arguments to run a ValidationOperator: (there are versions for Pandas, Spark and SQLAlchemy)

1 Like