GE on GCP Dataproc cluster

Archive
1

Hi all! :wave:

I am working on implementing the GE library on Dataproc clusters , programmed by Composer (Airflow), have any of you implemented this? maybe we can share tips and settings.

for example, I currently have a issue:

raise VersionConflict(dist, req).with_context(dependent_req)

pkg_resources.ContextualVersionConflict: (google-resumable-media 0.7.1 (/opt/conda/miniconda3/lib/python3.7/site-packages), Requirement.parse(‘google-resumable-media<0.6dev,>=0.5.0’), {‘google-cloud-bigquery’})

I installed google-resumable-media==0.5.0 but the issue continue, any idea?