# How to organize validation results comes from multiple pipelines?

**URL:** <https://discourse.greatexpectations.io/t/how-to-organize-validation-results-comes-from-multiple-pipelines/353>\
**Category:** Archive\
**Created:** [August 27, 2020, 10:06pm UTC](https://discourse.greatexpectations.io/t/how-to-organize-validation-results-comes-from-multiple-pipelines/353 "2020-08-27T22:06:36Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![dennys](https://yyz1.discourse-cdn.com/flex031/user_avatar/discourse.greatexpectations.io/dennys/32/74_2.png) [@dennys](https://discourse.greatexpectations.io/u/dennys)\
**Post date:** [August 27, 2020, 10:06pm UTC](https://discourse.greatexpectations.io/t/how-to-organize-validation-results-comes-from-multiple-pipelines/353/1 "2020-08-27T22:06:36Z")

</div>

I just wanna ask you about a known pattern to store validations/data-docs generated by multiple pipelines?  
My first guess is use different prefixes in the same bucket, something like this:

pipeline/simple-pipeline/validations  
pipeline/simple-pipeline/data-docs/site  
pipeline/other-pipeline/validations  
pipeline/other-pipeline/data-docs/site

The only trade off will be **not to have an html file to index** all pipelines docs

---

<div class="post-metadata">

**Author:** ![eugene.mandel](https://yyz1.discourse-cdn.com/flex031/user_avatar/discourse.greatexpectations.io/eugene.mandel/32/22_2.png) [@eugene.mandel](https://discourse.greatexpectations.io/u/eugene.mandel)\
**Post date:** [August 28, 2020, 3:53pm UTC](https://discourse.greatexpectations.io/t/how-to-organize-validation-results-comes-from-multiple-pipelines/353/2 "2020-08-28T15:53:34Z")

</div>

I see at least two approaches:

If each pipeline has its own Great Expectations Data Context (with its own `great_expectations.yml` config file), then the approach described in the question is the way to go: configure each Data Docs site to write to a different prefix in the same S3 bucket.

The second approach is for the two pipelines to share a Data Context (one `great_expectations.yml` config file). When you validate data by running a Validation Operator (or a Checkpoint), you pass `data_asset_name`. As long as these names in the two pipelines do not collide, the validation results coming from both can co-exist in the same Data Docs site.

---

<div class="post-metadata">

**Author:** ![dennys](https://yyz1.discourse-cdn.com/flex031/user_avatar/discourse.greatexpectations.io/dennys/32/74_2.png) [@dennys](https://discourse.greatexpectations.io/u/dennys)\
**Post date:** [August 28, 2020, 4:16pm UTC](https://discourse.greatexpectations.io/t/how-to-organize-validation-results-comes-from-multiple-pipelines/353/3 "2020-08-28T16:16:03Z")

</div>

Interesting…I like the second approach, maybe I can use the pipeline name as prefix to avoid collisions.

---

<div class="post-metadata">

**Author:** ![dennys](https://yyz1.discourse-cdn.com/flex031/user_avatar/discourse.greatexpectations.io/dennys/32/74_2.png) [@dennys](https://discourse.greatexpectations.io/u/dennys)\
**Post date:** [September 24, 2020, 8:13pm UTC](https://discourse.greatexpectations.io/t/how-to-organize-validation-results-comes-from-multiple-pipelines/353/4 "2020-09-24T20:13:13Z")

</div>

I’m getting `E TypeError: run() got an unexpected keyword argument 'data_asset_name'`  
trying to run the following:

```auto
results = context.run_validation_operator(
        assets_to_validate=[batch],
        run_id=airflow_run_id,        
        validation_operator_name="run_warning_and_failure_expectation_suites",
        data_asset_name=f'{ras_code}_{model}'
    )

```

Where I should pass ` data_asset_name` value?

---

<div class="post-metadata">

**Author:** ![eugene.mandel](https://yyz1.discourse-cdn.com/flex031/user_avatar/discourse.greatexpectations.io/eugene.mandel/32/22_2.png) [@eugene.mandel](https://discourse.greatexpectations.io/u/eugene.mandel)\
**Post date:** [September 25, 2020, 11:00pm UTC](https://discourse.greatexpectations.io/t/how-to-organize-validation-results-comes-from-multiple-pipelines/353/5 "2020-09-25T23:00:20Z")

</div>

run\_validation\_operator does not accept the data\_asset\_name argument.

I assume that you obtained the batch that you are passing to the Validation Operator by calling

> batch = context.get\_batch(batch\_kwargs, expectation\_suite\_name)

Then you should set a value of “`data_asset_name`” in your batch\_kwargs to the string you want Data Docs to use as the data asset name.

---

<div class="post-metadata">

**Author:** ![rubenssoto](https://avatars.discourse-cdn.com/v4/letter/r/e47774/32.png) [@rubenssoto](https://discourse.greatexpectations.io/u/rubenssoto)\
**Post date:** [November 3, 2020, 8:30pm UTC](https://discourse.greatexpectations.io/t/how-to-organize-validation-results-comes-from-multiple-pipelines/353/6 "2020-11-03T20:30:41Z")

</div>

> [@eugene.mandel](#):
>
> data\_asset\_name

I am having trouble with many expectations with the same data docs, I have two pipelines with GE and every run my data docs is overwritten.

Could you help me?