# How do I programmatically validate expectations?

**URL:** <https://discourse.greatexpectations.io/t/how-do-i-programmatically-validate-expectations/743>\
**Category:** Archive\
**Created:** [May 5, 2021, 7:04pm UTC](https://discourse.greatexpectations.io/t/how-do-i-programmatically-validate-expectations/743 "2021-05-05T19:04:08Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![ashwin153](https://avatars.discourse-cdn.com/v4/letter/a/b77776/32.png) [@ashwin153](https://discourse.greatexpectations.io/u/ashwin153)\
**Post date:** [May 5, 2021, 7:04pm UTC](https://discourse.greatexpectations.io/t/how-do-i-programmatically-validate-expectations/743/1 "2021-05-05T19:04:08Z")

</div>

I would like to create a function, `validate(df: pyspark.sql.DataFrame, expectations: List[great_expectations.expectations.Expectation]) -> None` that validates the `expectations` on the `df`. How would I go about implementing this function? After browsing the documentation and the codebase a little, I think I need to convert the `df` to a `Batch` using a `DataContext`, bind the `expectations` to a `ExpectationConfiguration`, and then validate the `ExpectationConfiguration` over the `Batch`. Does this seem reasonable? Is there a function somewhere in the library that already does this?

---

<div class="post-metadata">

**Author:** ![ashwin153](https://avatars.discourse-cdn.com/v4/letter/a/b77776/32.png) [@ashwin153](https://discourse.greatexpectations.io/u/ashwin153)\
**Post date:** [May 5, 2021, 9:36pm UTC](https://discourse.greatexpectations.io/t/how-do-i-programmatically-validate-expectations/743/2 "2021-05-05T21:36:02Z")

</div>

Also, completely tangential question, but how did you guys get the notebook auto-completion in your examples (e.g., [How to quickly explore Expectations in a notebook — great\_expectations documentation](https://docs.greatexpectations.io/en/latest/guides/tutorials/explore_expectations_in_a_notebook.html)) to work?

---

<div class="post-metadata">

**Author:** ![ashwin153](https://avatars.discourse-cdn.com/v4/letter/a/b77776/32.png) [@ashwin153](https://discourse.greatexpectations.io/u/ashwin153)\
**Post date:** [May 6, 2021, 12:56am UTC](https://discourse.greatexpectations.io/t/how-do-i-programmatically-validate-expectations/743/3 "2021-05-06T00:56:17Z")

</div>

I have made a little bit of progress. I think I need to do something like this.

```python
def validate(df, expectations) -> None:
    with tempfile.TemporaryDirectory() as temp:
        data_context_config = ge.data_context.types.base.DataContextConfig(
            datasources={
                "spark": {
                    "data_asset_type": {
                        "class_name": "SparkDFDataset",
                        "module_name": "great_expectations.dataset",
                    },
                    "class_name": "SparkDFDatasource",
                    "module_name": "great_expectations.datasource",
                    "batch_kwargs_generators": {},
                },
            },
            stores=ge.data_context.types.base.FilesystemStoreBackendDefaults(
                root_directory=temp,
            ))

        data_context = ge.data_context.BaseDataContext(
            project_config=data_context_config)
            
        expectation_suite = data_context.create_expectation_suite(
            expectation_suite_name="expectation_suite")

        batch = data_context.get_batch(
            batch_kwargs={
                "dataset": event.data.to_spark(),
                "datasource": "spark"
            },
            expectation_suite_name="expectation_suite")

```

The part I’m struggling with is how to run the expectations on the batch. Do you guys have any suggestions? The documentation makes it seem like you have to call the expectation functions on the batch, but I want to create the expectations and then call them on the batch later.

---

<div class="post-metadata">

**Author:** ![eugene.mandel](https://yyz1.discourse-cdn.com/flex031/user_avatar/discourse.greatexpectations.io/eugene.mandel/32/22_2.png) [@eugene.mandel](https://discourse.greatexpectations.io/u/eugene.mandel)\
**Post date:** [May 17, 2021, 4:33pm UTC](https://discourse.greatexpectations.io/t/how-do-i-programmatically-validate-expectations/743/4 "2021-05-17T16:33:19Z")

</div>

This how-to guide shows how to validate a Spark dataframe: [How to load a Spark DataFrame as a Batch — great\_expectations documentation](https://docs.greatexpectations.io/en/latest/guides/how_to_guides/creating_batches/how_to_load_a_spark_dataframe_as_a_batch.html)  
I think you can assemble your function from its parts.