Hey Guys, I was trying to migrate GX from 0.16.5 and am not being able to, cause apparently since GX 0.16.12 supposedly there was a fix for the persist parameter to work. The thing is that I wanted it to be False, but it seems the parameter is not taking effect.
Anyone facing similar problems?
I was following this guide How to connect to in-memory data in a Spark dataframe | Great Expectations, which seems to have been removed. Also tried with fluent approach and still failing.
Is the property persist being considered/passed? Shouldn’t it be one of the parameters in add_or_update_spark ?
Created this PR to try to address it. [BUGFIX] Fixed persist to be considered for spark fluent datasource by jmcorreia · Pull Request #8512 · great-expectations/great_expectations · GitHub