Connect GX to Athena is extremely slow

Hi there,

we use GX 0.16.16 to connect to Athena data source, however, the GX context loading time is super slow, it takes 6 mins on average to get the context and continue with the validation, I wonder does the team know what is causing the slowness in Athena connection? And if there is any recommendations to speed up this data source connection a bit?

Thank you in advance

There are a few reasons why Athena connections could be slow when utilising Great Expectations:

  • Data Volume and Complexity
  • Configuration and Optimization
  • Resource Limitations
  • Query Optimization

https://discourse.greatexpectations.io

Optimising data queries, enhancing configuration parameters, and making sure the infrastructure is sufficiently provisioned to handle the necessary data quantities and query complexity are frequently used in conjunction to address these problems.