Data Assistant always suggesting only one expectation

GX OSS Support
1

Hello,
I’ve tried with different datasets but always the data assistant is suggesting only this expectation:
expect_column_values_to_be_null
No matter dataset size, columns…always that. I would expect Data Assistant would suggest a lot of different expectations… This is the code used:

df = spark.table(“my_dataset_table”)
context.add_or_update_expectation_suite(“my_expectation_suite”)
validator = context.get_validator(
batch_request=my_batch_request,
expectation_suite_name=“my_expectation_suite”,
)
data_assistant_result = context.assistants.missingness.run(
validator=validator,
estimation=“flag_outliers”
)
data_assistant_result.plot_expectations_and_metrics()

2

The context.assistants.missingness is the Missingess Data Assistant, which only checks for missing values (nulls) in your data and creates Expectations related to that.

You can try the Onboarding Data Assistant with context.assistants.onboarding which generates a wider range of Expectations.

There’s also the Volume Data Assistant (context.assistants.volume) which creates Expectations for row count and the number of distinct values in a column.

I couldn’t find documentation for the Onboarding or Volume Data Assistants but they work the same way as the Missingness Assistant