GE with Kedro, load expectations from YAML

Archive
1

Hi,

Just started out using GE with Kedro and I have a question in how to load expectations using YAML.

This is what I have:

	# nodes/int_titanic.py

	def int_titanic(raw_df: pd.DataFrame, expectations_dict:Dict) -> pd.DataFrame:
	    """
	    Intermediate layer for titanic.
	    Args:
	        raw_df (DataFrame): Raw DataFrame from titanic.csv.

	    Returns:
	        DataFrame cleaned.

	    """
	    ge_df = ge.dataset.PandasDataset(raw_df)

	   # I want to be able to make this or something similar work
	    result = ge_df.validate(expectations_config=expectations_dict)


	    # Check result
	    if not result.success:
	        print(result)
	        raise Exception('Validation error.')

	    return raw_df

	# conf/base/parameters.yml

	int_titanic_expectations:
	  expectations:
	    -
	      kwargs:
	        column: Name
	        type_: str
	      expectation_type: expect_column_values_to_be_of_type
	    -
	      kwargs:
	        column: Age
	        type_: float
	      expectation_type: expect_column_values_to_be_of_type
2

Thanks, @tatianassaturno!

As we said on the hangout, you can resolve it by typing the expectations_dict:

result = ge.validate(
    ge_df,
    expectation_suite=ge.core.ExpectationSuite(expectations_dict)
)

I believe this will also work:

ge_df.validate(ge.core.ExpectationSuite(expectations_dict))
1 Like
3

Yes, it’s working fine. Thank you!

4

I had to add ** before expectations_dict.