Issue Initializing BigQuery Datasource in Great Expectations

GX Core Support
1

Hi GE Community,

I’m trying to set up Great Expectations with BigQuery as my datasource, but I keep running into the following error:

python

CopyEdit



raise gx_exceptions.DatasourceError(
great_expectations.exceptions.exceptions.DatasourceError: Cannot initialize datasource my_bigquery_datasource, error: The given datasource could not be retrieved from the DataContext; please confirm that your configuration is accurate.

What I Have Tried:

  • Verified that my great_expectations.yml configuration includes the correct BigQuery connection details.
  • Ensured that my Google Cloud credentials are properly set up and gcloud auth works.
  • Tried creating the datasource manually in a Python script using:

python

CopyEdit

from great_expectations.data_context import DataContext

context = DataContext()

context.add_datasource(
name=“my_bigquery_datasource”,
class_name=“Datasource”,
execution_engine={
“class_name”: “SqlAlchemyExecutionEngine”,
“connection_string”: “bigquery://my-project-id”
},
data_connectors={
“default_runtime_data_connector”: {
“class_name”: “RuntimeDataConnector”,
“batch_identifiers”: [“batch_id”],
}
}
)

Issue Details:

  • The error suggests that GE cannot retrieve the datasource from the DataContext.
  • I suspect an issue with how GE initializes the datasource.

Any guidance on troubleshooting this issue would be greatly appreciated! Has anyone faced a similar problem when setting up BigQuery as a datasource?

Thanks in advance!