ExpectColumnDistinctValuesToEqualSet & its output

GX Core Support
1

Hi, I am using the following expectation to check a column = workitem_type which should ONLY have value = Bug.

“”"

EXPECTATION#1: Column = workitem_type (type = Bug only)

Scope : Table-Specific (BUGS)
Business Rule : workitem_type column must strictly contain ONLY ‘Bug’.
Target Column : workitem_type
Validation : Value set = [‘Bug’]
Failure Impact: Critical - Unexpected workitem types present in dataset.

“”"

exp_workitem_type_is_bug = gx.expectations.ExpectColumnDistinctValuesToEqualSet(
column=“workitem_type”,
value_set=[“Bug”]
)

The script correctly fails this expectation & returns

{

  "success": false,

  "expectation_config": {

    "type": "expect_column_distinct_values_to_equal_set",

    "kwargs": {

      "batch_id": "gx_sample_data_source-BUGS_asset",

      "column": "workitem_type",

      "value_set": \[

        "Bug"

      \]

    },

    "meta": {},

    "id": "97fd3cfe-8648-47b8-ab09-7dc65899c488",

    "severity": "critical"

  },

  "result": {

    "observed_value": null,

    "unexpected_count": 1,

    "missing_count": 0,

    "partial_unexpected_list": \[

      "Defect"

    \],

    "partial_missing_list": \[\]

  },

  "meta": {},

  "exception_info": {

    "raised_exception": false,

    "exception_traceback": null,

    "exception_message": null

  }

}

In my dataset there are 7 rows which has “Defect” instead of “Bug”. I am interested in reporting “7”. Which expectation can help me achieve that. If there is none, then how to use “ExpectColumnDistinctValuesToEqualSet” to get the value = 7.

2

Hi @KK1259, thanks for the question. The mismatch here is that ExpectColumnDistinctValuesToEqualSet is an aggregate Expectation - it builds a set from all the values found in that column, and then validates that the resulting set is equal to parameter value_set. You’re describing a row-level check, so the Expectation you’re looking for is ExpectColumnValuesToBeInSet. The result includes partial_unexpected_counts, which for your example data would return [{"value": "Defect", "count": 7}]. Hope that helps!

3

Thank you @joshstauffer , that worked !