# Appending validations to data docs

**URL:** <https://discourse.greatexpectations.io/t/appending-validations-to-data-docs/1404>\
**Category:** GX Core Support\
**Created:** [September 25, 2023, 6:44am UTC](https://discourse.greatexpectations.io/t/appending-validations-to-data-docs/1404 "2023-09-25T06:44:30Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![erman](https://avatars.discourse-cdn.com/v4/letter/e/94ad74/32.png) [@erman](https://discourse.greatexpectations.io/u/erman)\
**Post date:** [September 25, 2023, 6:44am UTC](https://discourse.greatexpectations.io/t/appending-validations-to-data-docs/1404/1 "2023-09-25T06:44:30Z")

</div>

Hi everyone,  
I am initializing a context and adding data sources and assets the following way:

```auto
config_data_docs_sites = {
        "s3_site": {
            "class_name": "SiteBuilder",
            "store_backend": {
                "class_name": "TupleS3StoreBackend",
                "bucket": "great-expectations",
                "prefix": "data_docs",
                "boto3_options": BOTO3_OPTIONS
            },
        },
    }
    data_context_config = DataContextConfig(
        store_backend_defaults=S3StoreBackendDefaults(default_bucket_name=GX_BUCKET_NAME),
        data_docs_sites=config_data_docs_sites
    )
    context = BaseDataContext(project_config=data_context_config)

 datasource_name = f"gx_temp_{latest_filename.replace('/', '_')}_{latest_version}"
    bucket_name = TEMP_DESTINATION_BUCKET
    datasource = context.sources.add_or_update_pandas_s3(
        name=datasource_name, bucket=bucket_name, boto3_options=BOTO3_OPTIONS
    )

for file_name_df in dataasset_filelist:
        file_name_df = file_name_df.replace('/', '_')
        if file_name_df.endswith('df1'):
            asset_name = f"{latest_filename}_{latest_version}_df1_{file_name_df}"
            s3_prefix = f"{latest_filename}/{latest_version}"
            batching_regex = file_name_df.split("_")[2] + "_" + file_name_df.split("_")[3]
            datasource.add_parquet_asset(name=asset_name, s3_prefix=s3_prefix, batching_regex=batching_regex)
        elif file_name_df.endswith('df2'):
            asset_name = f"{latest_filename}_{latest_version}_df2_{file_name_df}"
            s3_prefix = f"{latest_filename}/{latest_version}"
            batching_regex = file_name_df.split("_")[2] + "_" + file_name_df.split("_")[3]
            datasource.add_parquet_asset(name=asset_name, s3_prefix=s3_prefix, batching_regex=batching_regex)
        elif file_name_df.endswith('df3'):
            asset_name = f"{latest_filename}_{latest_version}_df3_{file_name_df}"
            s3_prefix = f"{latest_filename}/{latest_version}"
            batching_regex = file_name_df.split("_")[2] + "_" + file_name_df.split("_")[3]
            datasource.add_parquet_asset(name=asset_name, s3_prefix=s3_prefix, batching_regex=batching_regex)
        elif file_name_df.endswith('df4'):
            asset_name = f"{latest_filename}_{latest_version}_df4_{file_name_df}"
            s3_prefix = f"{latest_filename}/{latest_version}"
            batching_regex = file_name_df.split("_")[2] + "_" + file_name_df.split("_")[3]
            datasource.add_parquet_asset(name=asset_name, s3_prefix=s3_prefix, batching_regex=batching_regex)

validations_df1 = []
    validations_df2 = []
    validations_df3 = []
    validations_df4 = []
    for asset_name in asset_names:
        data_asset = data_source.get_asset(asset_name)
        my_batch_request = data_asset.build_batch_request()

        if asset_name.endswith('df1'):
            expectation = 'Exp_Abteilung'
            validations_df1.append(
                {"batch_request": my_batch_request, "expectation_suite_name": expectation}
            )
        elif asset_name.endswith('df2'):
            expectation = 'Exp_Person'
            validations_df2.append(
                {"batch_request": my_batch_request, "expectation_suite_name": expectation}
            )
        elif asset_name.endswith('df3'):
            expectation = 'Exp_Ausruestung'
            validations_df3.append(
                {"batch_request": my_batch_request, "expectation_suite_name": expectation}
            )
        else:
            expectation = 'Exp_Combined'
            validations_df4.append(
                {"batch_request": my_batch_request, "expectation_suite_name": expectation}
            )

checkpoint1 = context.add_or_update_checkpoint(
        name='checkpoint_df1',
        validations=validations_df1,
        run_name_template=f'{latest_file_name}_{latest_version}_df1',
        action_list=[
            {
                'name': 'store_validation_result',
                'action': {'class_name': 'StoreValidationResultAction'}
            },
            {
                'name': 'store_evaluation_params',
                'action': {'class_name': 'StoreEvaluationParametersAction'}
            },
            {
                'name': 'update_data_docs',
                'action': {'class_name': 'UpdateDataDocsAction'
                           }
            }
        ]
    )

results1 = checkpoint1.run()

```

GX is updating the data docs after each validation in the list validations\_df1 has been validated, which leads to very slow execution times. I am searching for a way to first validate all the validatiions in the list and only then to update the data docs to include the new validations together with the old ones, without rebuilding the entire data docs from scratch. I need to have all validations visible in data docs but cannot afford long runtimes.}  
Thank you for your help.

---

<div class="post-metadata">

**Author:** ![slack\_user](https://yyz1.discourse-cdn.com/flex031/user_avatar/discourse.greatexpectations.io/slack_user/32/202_2.png) [@slack\_user](https://discourse.greatexpectations.io/u/slack_user)\
**Post date:** [September 25, 2023, 11:10am UTC](https://discourse.greatexpectations.io/t/appending-validations-to-data-docs/1404/2 "2023-09-25T11:10:28Z")

</div>

Remove the third actions from your checkpoints action\_list `update_data_docs` and only at the end of your script/pipeline run python command `context.build_data_docs()` or on the CLI `great_expectations docs_build`  
Note: `@Lauri Huhta` originally [posted this reply in Slack](https://greatexpectationstalk.slack.com/archives/CUTCNHN82/p1695640220240969?thread_ts=1695624302.617329&cid=CUTCNHN82). It might not have transferred perfectly.

---

<div class="post-metadata">

**Author:** ![erman](https://avatars.discourse-cdn.com/v4/letter/e/94ad74/32.png) [@erman](https://discourse.greatexpectations.io/u/erman)\
**Post date:** [September 25, 2023, 11:12am UTC](https://discourse.greatexpectations.io/t/appending-validations-to-data-docs/1404/3 "2023-09-25T11:12:01Z")

</div>

```auto
context.build_data_docs()

```

this rebuilds the data docs completely. I need something to append the new validations to the existing data docs. Fully rebuilding the docs will get slow as more validations pile on.

---

<div class="post-metadata">

**Author:** ![nevintan](https://yyz1.discourse-cdn.com/flex031/user_avatar/discourse.greatexpectations.io/nevintan/32/272_2.png) [@nevintan](https://discourse.greatexpectations.io/u/nevintan)\
**Post date:** [September 25, 2023, 9:10pm UTC](https://discourse.greatexpectations.io/t/appending-validations-to-data-docs/1404/4 "2023-09-25T21:10:21Z")

</div>

Hi erman, I’m not sure if this is compatible with your use case, but have you looked into using UpdateDataDocsAction and specifying a list of sites via the `site_names` paramenter?

---

<div class="post-metadata">

**Author:** ![erman](https://avatars.discourse-cdn.com/v4/letter/e/94ad74/32.png) [@erman](https://discourse.greatexpectations.io/u/erman)\
**Post date:** [September 26, 2023, 7:09am UTC](https://discourse.greatexpectations.io/t/appending-validations-to-data-docs/1404/5 "2023-09-26T07:09:49Z")

</div>

hi nevintan,  
yes i tried it but it’s the same as if i don’t specify a site name.