# Can I use a Spark Dataframe with Checkpoints?

**URL:** <https://discourse.greatexpectations.io/t/can-i-use-a-spark-dataframe-with-checkpoints/797>\
**Category:** Archive\
**Created:** [June 22, 2021, 8:49am UTC](https://discourse.greatexpectations.io/t/can-i-use-a-spark-dataframe-with-checkpoints/797 "2021-06-22T08:49:36Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![claudiahuber](https://avatars.discourse-cdn.com/v4/letter/c/e47c2d/32.png) [@claudiahuber](https://discourse.greatexpectations.io/u/claudiahuber)\
**Post date:** [June 22, 2021, 8:49am UTC](https://discourse.greatexpectations.io/t/can-i-use-a-spark-dataframe-with-checkpoints/797/1 "2021-06-22T08:49:36Z")

</div>

Hi,

can I use a runtime batch request I defined with a spark data frame with checkpoints? Or is this currently unsupported?

The configuration I have works fine with validation operators but I could not figure out with the documentation how / if this could be done with the newer checkpoints.

I followed this guide for V3 to define the batch request:

[https://docs.greatexpectations.io/en/latest/guides/how\_to\_guides/creating\_batches/how\_to\_load\_a\_spark\_dataframe\_as\_a\_batch.html](https://docs.greatexpectations.io/en/latest/guides/how_to_guides/creating_batches/how_to_load_a_spark_dataframe_as_a_batch.html)

> runtime\_batch\_request = RuntimeBatchRequest(  
> datasource\_name=“my\_spark\_datasource”,  
> data\_connector\_name=“my\_runtime\_data\_connector”,  
> data\_asset\_name=“insert\_your\_data\_asset\_name\_here”,  
> runtime\_parameters={  
> “batch\_data”: df  
> },  
> batch\_identifiers={  
> “some\_key\_maybe\_pipeline\_stage”: “ingestion step 1”,  
> “some\_other\_key\_maybe\_airflow\_run\_id”: “run 18”  
> }  
> )