How to add all batch partitions to checkpoint

Archive
1

I am wondering how to simply add all the partitions specified by the datasource to the checkpoint.
it seems odd to copy and paste the batch definition over and over, changing the index.

Checkpoint yml

name: carriers
config_version: 1
class_name: SimpleCheckpoint
validations:
  - batch_request:
      datasource_name: db
      data_connector_name: random
      data_asset_name: Carrier_THG_20200324
      partition_request:
         partition_identifiers:
            RandomId: 0

    expectation_suite_name: carrier

config datasources yml

datasources:
  db:
    module_name: great_expectations.datasource
    connection_string: mssql+pyodbc://SERVER/DATABASE?driver=ODBC+Driver+17+for+SQL+Server&charset=utf&autocommit=true&Trusted_Connection=yes
    tables:
      TABLENAME:
        partitioners:
          random:
            splitter_method: _split_on_mod_integer
            splitter_kwargs:
              column_name: RandomId
              mod: 7
    class_name: SimpleSqlalchemyDatasource
2

@hansen.peterd This is a great idea to have a fast way to configure “all partitions”. Could you please copy this into a GitHub issue so that we can track it as a feature request? Thank you!

3

Here is the issue

1 Like
4

Prefect - thank you for filing it!