I would like to use GE pre-0.13 release to create custom expectations for a SparkDFDataset. I’m working in an Azure Databricks Notebook environment and I have a pre-existing data pipeline which loads data from my data lake into a spark dataframe then performs custom business validations. I’m looking for a code example which shows me how create, instantiate and use custom expectations on my spark dataframe.
1 Like