Great Expectations has a new home: dbt/Fivetran has taken stewardship of the library and is investing in its future. Over the last few years, OSS development slowed as GX Labs focused on GX Cloud, their commercial offering. With dbt/Fivetran’s backing, GX Core now has dedicated, full-time maintainership, and the roadmap is community-focused. Contribution docs have been thoroughly revamped, and thoughtful PRs are welcome across the codebase.
The first priority on the roadmap is hardening existing capabilities. The GX community has documented many gaps over the years, some of which were simply not prioritized. We’ve analyzed old issues — many of which were closed as “won’t fix” — and clustered them to identify which areas of the codebase need the most work. The test suite is being expanded, and data sources marked as “experimental” are gaining proper coverage.
The v1 release removed Data Assistants and Profilers, with the promise that data profiling and automated configuration would return in the future. That time is now — the next GX release (1.21.0) ships with agent skills, packaged workflows that let users configure Data Sources and Expectations with their LLM of choice, and upcoming work will support automatically generating Expectations from data.
Custom Expectations will return! GX v1 removed the capability to write custom Expectations, in part because the underlying metrics layer has significant complexity. Future work will use the new metrics API to reintroduce Custom Expectations, allowing users to freely define their own metrics and Expectations as plugins.
There are many other parts of the library that we’re looking forward to improving, including Data Docs, Validation Results, broader Data Source support, and more supported Expectations.
If you use Great Expectations, we want your input — and the GX Community Meetup is back: Thursday, August 20, 11:00 ET / 17:00 CEST, on Zoom. Join us to catch up on these changes, see a demo of upcoming features, and weigh in on the roadmap. Can’t make it? Join the conversation in GitHub Discussions — we’re building the future of the library together.